Equilibrium thermodynamics in modified gravitational theories 
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We show that it is possible to obtain a picture of equilibrium thermodynamics on the apparent 
horizon in the expanding cosmological background for a wide class of modified gravity theories with 
the Lagrangian density f{R,(t>,X), where R is the Ricci scalar and X is the kinetic energy of a 
scalar field 0. This comes from a suitable definition of an energy momentum tensor of the "dark" 
component that respects to a local energy conservation in the Jordan frame. In this framework 
the horizon entropy S corresponding to equilibrium thermodynamics is equal to a quarter of the 
horizon area A in units of gravitational constant G, as in Einstein gravity. For a flat cosmological 
background with a decreasing Hubble parameter, 5* globally increases with time, as it happens for 
viable f{R) inflation and dark energy models. We also show that the equilibrium description in 
terms of the horizon entropy S is convenient because it takes into account the contribution of both 
the horizon entropy S in non-equilibrium thermodynamics and an entropy production term. 



I. INTRODUCTION 

The discovery of black hole entropy by Bekenstein 
opened up a window for a profound physical connection 
between gravity and thermodynamics The gravita- 
tional entropy S in Einstein gravity is proportional to 
the horizon area A of black holes, such that S = A/ {AG), 
where G is gravitational constant. A black hole with 
mass M obeys the first law of thermodynamics, TdS = 
dM 0, where T = |Ks|/(27r) is a Hawking temperature 
determined by the surface gravity Kg 

Since black hole solutions follow from Einstein field 
equations, the first law of black hole thermodynamics 
implies some connection between thermodynamics and 
Einstein equations. In fact Jacobson Q showed that 
Einstein equations can be derived by using the Clau- 
sius relation TdS* = dQ on all local acceleration hori- 
zons in the Rindler space-time together with the relation 
S cc A, where dQ and T are the energy flux across the 
horizon and the Unruh temperature seen by an acceler- 
ating observer just inside the horizon, respectively. This 
approach was applied to a number of cosmological set- 
tings such as quasi dc Sitter inflationary universe Q 
and dark energy dominated universe 0. 

Unlike stationary black holes the expanding universe 
has a dynamically changing apparent horizon. For dy- 
namical black holes. Hay ward Q developed a frame- 
work to deal with the first law of thermodynamics on 
a trapping horizon in Einstein gravity (see Ref. Q for 
related works). This was extended to the Friedmann- 
Lemaitre- Robertson- Walker (FLRW) space-time [l^l, in 
which the Fricdmann equation can be written in the form 
TdS = —dE + WdV, where E is the intrinsic energy and 
W is the work density present in the dynamical back- 
ground. For matter contents of the universe with energy 
density p and pressure P, the work density is given by 
W = {p — P)/2. Note that the energy fiux dQ in the Ja- 
cobson's formalism is equivalent to —dE + WdV in the 
FLRW background. 

In the theories where the Lagrangian density / is a 



non-linear function in terms of the Ricci scalar R (so 
called "/(i?) gravity"), Eling et al. pointed out that 



a non-equilibrium treatment is required such that the 
Clausius relation is modified to dS = dQ/T -\- diS. Here 
the horizon entropy is defined by S* = F{R)A/ (AG) with 
F{R) ~ df/dR and diS describes a bulk viscosity en- 
tropy production term. The variation of the quantity 
F{R) gives rise to the non-equilibrium term diS that is 
absent in Einstein gravity, where a hat denotes the quan- 
tity in the non-equilibrium thermodynamics. 

The connections between thermodynamics and mod- 
ified gravity have been extensively discussed, including 
the theories such as f{R) gravity [l2l - [l6l |. scalar-tensor 
Gauss-Bonnet and Lovelock gravity 
raneworld models [1^ (see also Ref. plj). 
In Gauss-Bonnet and Lovelock gravity, it is possible to 
rewrite Einstein equations in the form of equilibrium 
thermodynamics [l7|. On the other hand, in f{R) grav- 
ity and scalar-tensor theories with the Lagr ang ian den- 
sity F{(j))R, it was shown in Refs. [H [11, [11 [13] that, by 
employing the Hayward's dynamical framework, a non- 
equilibrium description of thermodynamics arises for the 
Wald's horizon entropy defined by 5 = FA/{AG) 
This is consistent with the results obtained by Eling et 

al. [m. 

The appearance of a non-equilibrium entropy produc- 
tion term diS is intimately related to the theories in 
which the derivative of the Lagrangian density / with re- 
spect to R is not constant. It is of interest to see whether 
an equilibrium description of thermodynamics is possi- 
ble in such modified gravity theories. In this paper we 
will show that equilibrium thermodynamics exists for the 
general Lagrangian density f{R,(j),X), where / is func- 
tion of R, a scalar field 0, and a field kinetic energy X. 
This is possible by introducing the Bekcnstcin-Hawking 
entropy with a suitable redefinition of the "dark com- 
ponent" that respects a local energy conservation. This 
approach is convenient because the horizon entropy S 
defined in this framework involves the information of the 
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M) 



horizon entropy S in non-equilibrium thermodynamics 
together with the entropy production term. 



II. THERMODYNAMICS IN MODIFIED 
GRAVITY-NON-EQUILIBRIUM PICTURE 

We start with the following action 

16^ / J d^x^A/Cg^.,* 

(1) 

where g is the determinant of the metric tensor g^^,, Cm is 
the matter Lagrangian that depends on and matter 
fields *M, and X = - (l/2)5^''V^0Vi.0 is the kinetic 
term of a scalar field (j) (^/j is the covariant derivative 
operator associated with .g^ji/). The action ([Ij can de- 
scribe a wide variety of modified gravity theories, e.g., 
f{R) gravity, Brans-Dicke theories, scalar-tensor theo- 
ries, and dilaton gravity, in addition to quintessence and 
k-essence. 

From the action ([T]), the gravitational field equation 
and the equation of motion for (j> are derived as |24l | 

FG^,. = STrGT^f ' + ^g^,{f - RF) + V^V.F 



(2) 
(3) 



where F = df/OR, ^ df/dX, 



(2/y^),5£M/<5g^", and G^. = i?^. - (1/2) 5^-^? is the 
Einstein tensor. For the matter energy momentum ten- 
sor T^^^' we consider perfect fiuids of ordinary matter 
(radiation and non-relativistic matter) with total energy 
density pf and pressure Pf. 

We assume the 4-dimensional Fricdmann-Lemaitre- 
Robertson- Walker (FLRW) space-time with the metric, 

ds^ ^ Kpdx°'dxf^ -f f^dn^ , (4) 

where f = a{t)r and = t,x^ = r with the 2- 
dimensional metric hap ~ diag(— 1, a^(t)/[l — Kr'^]). 
Here a{t) is the scale factor, K is the cosmic curvature, 
and dfl^ is the metric of 2-dimensional sphere with unit 
radius. In the background ([4]) we obtain the following 
field equations from Eqs. ([2]) and ([3]): 



3F {H^ + K/a^) 

= LxX + ^{FR-f)- 3HF + SnGpf 



-2F{h - K/a^ 
= f^xX + F-HF + SnGipf + Pf) , 



(5) 

(6) 
(7) 



where a dot represents a derivative with respect to cosmic 
time t and the scalar curvature is given by i? = 6{2H^ + 



H + K/a?). The perfect fluid satisfies the continuity 
equation 



Pf + iH{ps + Pf)^Q. 
Equations ([5]) and (O can be written as 



where 



(8) 

(9) 
(10) 

(11) 
(12) 



We use a hat to represent quantities in the non- 
equilibrium description of thermodynamics. Note that 
Pd and Pd originate from the energy-momentum tensor 
fl'i) defined by 
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SttG , ^ , 
= 3^ {Pd + Pf) , 






H - 


K 
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p ypd + Pd + Pf 
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SttG 
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SttG 


F + 2HF - \{FR- 
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SttG 



9t.M - RF) + V^ V.F - g^.UF 
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where the Einstein equation can be written as 
SttG 



G 111/ — 



rp{d) I rp 



{M) 



(13) 



(14) 



If we define the density pd and the pressure Pd of 
"dark" components in this way, we find that these obey 
the following equation 



Pd + 3H{pd + Pd 



SttG 



[H'^ + K/a^)F , 



(15) 



where we have used Eq. ([7]) . For the theories with F ^ Q 
the r.h.s. of Eq. ([15]) does not vanish, so that the stan- 
dard continuity equation does not hold. This happens 
for f{R) gravity and scalar-tensor theory. 

Let us proceed to the thcrmodynamical property of 
the theories given above. First of all the apparent hori- 
zon is determined by the condition h^^dafdpf = 0, which 
means that the vector Vf is null on the surface of the ap- 
parent horizon. For the FLRW space-time the radius of 

the apparent horizon is given by ~ (_ff ^ + K/a^) 
Taking the time derivative of this relation and using 
Eq. Uni) it follows that 



47rG 



Pd 



Pd 



Pf 



Pf]dt. 



(16) 



The Bekcnstcin-Hawking horizon entropy in the Ein- 
stein gravity is given by S* = A/ (4G), where A = Anf^ is 
the area of the apparent horizon [l|-|3 . In the context of 
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modified gravity theories, Wald introduced a horizon en- 
tropy S associated with a Noether charge p^ . [23j . The 
Wald entropy 5 is a local quantity defined in terms of 
quantities on the bifurcate Killing horizon. More specifi- 
cally, it depends on the variation of the Lagrangian den- 
sity of gravitational theories with respect to the Rie- 
mann tensor. This is equivalent to S = A/{AGcs), where 
Geff = G/F is the effective gravitational coupling j26l |. 
Using the Wald entropy 



AF 
4G 



(17) 



together with Eq. ([T5| . we obtain 



1 



2TTrA 



-dS = AnfiH 



Pd 



Pd 



Pf 



P. 



(18) 

The apparent horizon has the following Hawking temper- 
ature T = |ks |/(27r), where Kg is the surface gravity given 

by 



From Eqs. and ([23]) we obtain 

TdS ^ ~dE + 2TTf\{pd + Pf-Pd~Pf)dfA 

+^{l + 27TfAT)dF. (24) 

As in Refs. [1, [T^ we introduce the work density 

W ={pd + Pf-Pd-Pf)/2. (25) 
Then Eq. ([M]) reduces to 

TdS' = -d^ + WdV + ^ (1 + 2TTfAT) dF . (26) 
This equation can be written in the form 

TdS" + Td^S = -dE + WdV , (27) 



rA 



'fA 2HfA 
2ttG_ 



2H^ + — 



3F 



■TA 



[pT - 3Pt) 



(19) 



with pt = Pd + Pf and Pt = Pd + Pf- As long as the 
total equation of state wt — Pt/ Pt satisfies wt < 1/3 
it follows that Kg < 0, which is the case for standard 
cosmology. Hence the horizon temperature is 



T 



1 



2'Kr/ 



1 



rA 
2Hf, 



Multiplying the term 1 - fA/{2HfA) for Eq. ((T 
obtain 



(20) 



we 



TdS* = ^TTf\H{pd + Pd + Pf + Pf)dt 



-2'Kr\{pd + Pd + Pf 



T 



Pj)dfA + ^TTfldF 



(21) 



In Einstein gravity the Misner-Sharp energy 125[ is 
defined to be ii^ = f^/(2G'). In /(i?) gravity and 
scalar-tensor theory this was extended to the form 
E = fAF/{2G) [IJ]. Using this latter expression for 
f{R,4>,X) theories, it follows that 



^^rAF^ ^3F{H^ + K/a^) 



2G 



8ttG 



Vipd + Pf), (22) 



where V = 47rf^/3 is the volume inside the apparent 
horizon. Using Eqs. ([8]) and ^TE\\ . we find the following 
relation 



dE 



ATif\H{pd + Pd + Pf + Pf)dt 



+^TTr\{pd + Pf)drA 



rA 
2G^ 



dF. 



(23) 



where 



d^S 



T2G ^ 



2TifAT) dF : 




(28) 



The new term diS may be interpreted as a term of en- 
tropy production in the non-equilibrium thermodynam- 
ics. The theories with F = constant lead to diS = 0, 
which means that the the first-law of the equilibrium 
thermodynamics holds. Meanwhile the theories with 
dF 7^ 0, including f{R) gravity and scalar-tensor the- 
ory, give rise to the additional term ([^5]) . 

It is clear from Eq. (jTSj) that the density pd and the 
pressure Pd defined in Eqs. pT|) and p2)) do not satisfy 
the standard continuity equation for F ^Q. If it is possi- 
ble to define pd and Pd obeying the conserved equation, 
then we anticipate that the non-equilibrium description 
of thermodynamics may not be necessary. In the next 
section we shall show that such a treatment is indeed 
possible. 



III. EQUILIBRIUM INTERPRETATION OF 
THERMODYNAMICS IN MODIFIED GRAVITY 



Let us consider equilibrium description of thermody- 
namics for theories with the action ((T|). Equations ([5]) 
and ^ can be written as 



3 + =87rG(pd + p/) , (29) 
-2 ( F - ^) = 8^G {pd + Pd + Pf + Pf) , (30) 
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where 



1 



Pd 



8ttG ^ 
+3(l-F){H^ + K/a^ 



1 



F + 2HF-^{FR- f) 



8ttG ^ 

-(1 - F){2H + + K/a^ 



(31) 



(32) 



If wc define Pel and in this way, they obey the fohowing 
continuity equation 



Pd + iH{pd + Pd)^0., 



(33) 



where we have used Eq. 
is replaced by 



©. We then find that Eq. ^ 



AttG 



[H{pd + Pd + Pf + Pf)dt. 



(34) 



In the equihbrium description there is no need to de- 
fine Geff in the field equations in (|29p and (PU)) in con- 
trast to Eqs. ^ and (|10l) in the non-equilibrium descrip- 
tion. This originates from the redefinition of energy den- 
sity Pd and pressure Pd of "dark" components defined in 
Eqs. (PT|) and respectively. This redefinition leads to 
the continuity equation p3p . In other words, the energy- 
momentum conservation in terms of "dark" components 
is met. Since the perfect fiuid of ordinary matter also sat- 
isfies the continuity equation ([5]), the total energy density 
Pt = Pd + Pf and the total pressure Pt 
universe obey the continuity equation 



Pd + Pf of the 



PT + 3H{pT + Pt) = . 



(35) 



Hence the equilibrium treatment of thermodynamics can 
be executed similarly to that in Einstein gravity. As a 
conseciuence, we introduce the Bekenstein-Hawking en- 
tropy [IHi 



A 
AG 



1 



G + X/a2 



(36) 



unlike the Wald entropy associated with Goff in Eq. pT)) 
in the non-equilibrium thermodynamics. This allows us 
to obtain the equilibrium description of thermodynamics 
as that in Einstein gravity. Note that the Bekenstein- 
Hawking entropy (j36p is a global geometric quantity pro- 
portional to A, which is not directly affected by the dif- 
ference of gravitational theories (i.e. by the difference of 
the quantity F = df/dR). 

From the definition ((36|) it follows that 

-Vd5 = Anf\H {pa + Pd + pf + Pf)dt. (37) 
ZnrA 

Using the horizon temperature given in Eq. ([20)1 . wc ob- 
tain 

TdS = 4TTf\H{pd + Pd + Pf + Pf)dt 



Defining the Misner-Sharp energy to be 



E = ^ = Vipd + Pf), 



(39) 



we find 



dE = -A7rf\H{pd + Pd + Pf + Pf)dt + A7rf\{pd + pf)dfA ■ 

(40) 

Due to the conservation equation (|33|. the r.h.s. of 
Eq. ()40p does not include an additional term proportional 
to dF. Combing Eqs. ^ and (gHl) gives 



(41) 



TdS* = -dE + WdV , 
where the work density W is defined by 

W={pd + Pf-Pd-Pf)/2. 



(42) 



Equation (|4ip corresponds to the first law of equilibrium 
thermodynamics. This shows that the equilibrium form 
of thermodynamics can be derived by introducing the 
density pd and the pressure Pd in a suitable way. 
Plugging Eqs. ^ and ^ into Eq. (gT]), we find 



TS = vU-^ 



{Pd + Pf+Pd + Pf) 



(43) 



Using V ^ 47rfy3 and Eq. (gOl), it follows that 



6TrHVfA{pd + Pf + Pd 

2t: H{H - K/a^) 
^~G {H^ + K/a^y ■ 



Pf) 



(44) 



The horizon entropy increases as long as the null en- 
ergy condition pT + Pt = Pd + Pf + Pd + Pf > is 
satisfied. S decreases for the total equation of state 
wt = Pt I Pt < —1, as it happens in General Relativity. 
Meanwhile the Wald entropy P7)) does not in general 
possess this property, as we will see for /(i?) infiation 
models in Sec. IV Al 

The above equilibrium picture of thermodynamics is 
intimately related with the fact that there is an energy 
momentum tensor satisfying the local conservation 
law V^T^i^'' = 0. This corresponds to writing the Einstein 
equation in the form 



(45) 



where 



rj.(d) 



1 



SttG 
1 



/,xV^0V,0 + (1 - F)i?^, 



(46) 



-2Tir\{pd + Pd + Pf + Pf)drA ■ (38) and V'^T^f = 



Defining T^? in this way, the local conservation of T^? 
follows from Eq. (|45]) because of the relations V^G^^ = 

(A/) 
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One can show that the horizon entropy S in the equi- 
hbrium picture has the following relation with S in the 
non-equilibrium picture: 



dS = dS + diS 



dF 



2GT 

27r(l - F) H{H - K/o?) 
G + Kla?Y 



dt. (47) 



Using the relations (|28l) and Eq. gT]) reduces to the 
following form 



dS 



where 



d,S 



1 2H^ + H + K/a^ 



dtS , 



F 4iJ2 +H + m/a? 
67r 4ij2 + iJ + m/a"^ dF 



G m + Kjd^ 



R 



(48) 



(49) 



While S is identical to S in Einstein gravity (F = 1), 
the difference appears in modified gravity theories with 
dF 7^ 0. Equation (|48l) shows that the change of the hori- 
zon entropy S in the equilibrium framework involves the 
information of both dS and diS in the non-equilibrium 
framework. 



IV. EINSTEIN FRAME IN SCALAR-TENSOR 
THEORIES 

For some specific theories, the action ([T]) can be trans- 
formed to the so-called Einstein frame action via the con- 
formal transformation. For example let us consider the 
following scalar-tensor theories with the action 



d'^x^ 



l^R + uj{<P)X^V{<j,) 



d^x C]\i{gfj,u, *A/) , 



(50) 



where = SttG, and F{(p), uj{4>), V{(j)) are functions of 
(j). Under the conformal transformation, g^^i, = Fg^^we 
obtain the following action in the Einstein frame |27l . |28| : 



1 



Ie = J d'^x^/^ 

- [ d^xCM{F{(j))-^~gf,,,^M), 



(51) 



where a tilde represents quantities in the Einstein frame, 
and 




u 



V 

J2 



(52) 



Varying the action (|5ip with respect to if, we obtain the 
field equation 



1 dC 



M 



dip 



0. 



(53) 



In the FLRW background the following relations hold 



dt = \/Fdi, a = VFa. 



(54) 



Using the relation —OCm / dip 
Q{ip) EE -F^/(2kF) and Tm 
equation ([55)1 reduces to 



'~-gKQ{ip)TM, where 
-Pf + 3pf, the field 



<p + 3Hip + [7,^ = -Q{ip){pf - 3Pf) 



(55) 



In this section a dot represents a derivative with respect 
to i. Introducing p^p = •pP' jl + U(lp) and P,^ = '^^/2 — 
U{lp), Eq. can be written as 



+ 3Hip^ + P^) = -Qiip){pf - 3Pf)ip . 



(56) 



Note that pf and Pf in the Einstein frame are related 
with Pf and Pf in the Jordan frame via Pf = F^Pf and 
Pf = F^Pf. Using Eqs. (gl) and it then follows that 

^f + 3H{pf + Pf) = +Q{^){pf ~ iPf)^ ■ (57) 



Equations ((56|) and ((57l) show that the field Lp is cou- 
pled to matter with the coupling Q{lp) except for ra- 
diation [2^. In Brans-Dicke theory with F{(j)) = K(j) 
and Ljj{(j)) ~ Lo-QD / {ti(j}) (ojbd is a constant parameter), 
the coupling Q is a constant [l^. The f{R) gravity 
in the metric formalism corresponds to wbd = and 
V = {FR - /)/(2k2), in which case Kip = yjij2 In F 
from Eq. ([52]) . Hence the couphng Q{(p) = —F^^/{2kF) 
is also a constant {Q = — 1/V6) in metric f{R) gravity 

From Eqs. (f56| and ([57|l the total energy density pr = 
Ptp + Pf and the total pressure Pt = P^p + Pf satisfy the 
continuity equation, dpr/di + 3H{pT + Pt) ~ 0. The 
following equations also hold in the Einstein frame 



3{H' + K/a')^K^ (p^ + Pf) , 



(58) 



2{H - K/d^) = -K^{p^ + Pp + Pf + Pf) ■ (59) 
We define several thermodynamical quantities 

«' = {p,+P,-P^-P,}/2, (60) 

where = {H'^ + K/a?)~'^/'^, A ~ 4:'jTr\, and kg ~ 
— [1 — {dfA/dt)/{2HfA)]/rA- Following the similar pro- 
cedure as in Sec, mil we arrive at the first law of thermo- 
dynamics. 



TdS 



-dE + WdV . 



(61) 



Hence the equilibrium description of thermodynamics 
holds in the Einstein frame. 

In the following we shall find the relation of the equilib- 
rium description in the Jordan frame for the fiat universe 
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{K = 0). The horizon entropy and the Misner-Sharp en- 
ergy in the Einstein frame are given, respectively, by 



2GH 



(62) 



These are different from those in the equilibrium picture 
in the Jordan frame (5 = 7r/{GH^) and E ^ 1/{2GH)) 
because of the difference of the Hubble parameter: 



2F dt 



F[H + ^Q^] . (63) 



In the Jordan frame the following relations hold 



1 dH ^ 



(64) 



Note that similar relations hold in the Einstein frame 
by adding the tilde for corresponding quantities. Using 
Eq. ((63)) we obtain 



where 



and 



dE = {fi/VF)dE 



1 + KQ[ip + [Q.^/Q - kQ)^^ - Hip]/H 



(1 + KQip/ny 



WdV = J^^^±J^WdV , 



TdS 



/F{3 + H/H^) 
F{2 + H/H^) 



(65) 
(66) 

(67) 
(68) 



Recall that the field Lp in Eq. ([66]) satisfies Eq. ([55]). 

Since F ~ 1 and ^ = 1 in Einstein gravity, one has 
dE ^ dE, WdV = WdV, and TdS = fdS. In scalar- 
tensor theories in which F and fi dynamically change in 
time, the equilibrium description of thermodynamics in 
the Jordan frame is not identical to that in the Einstein 
frame. We note that general modified gravity theories 
with the action ([T]) do not necessarily have the action in 
the Einstein frame. Even for such general theories we 
have shown in Sec. IIIII that the equilibrium picture of 
thermodynamics is present without any reference to the 
Einstein frame. 

Moreover, we regard that the frame in which the 
baryons obey the standard continuity equation p„i oc 
a~^, i.e. the Jordan frame, is the "physical" frame where 
physical quantities are compared with observations and 
experiments. The direct construction of the equilibrium 
thermodynamics in the Jordan frame is not only versatile 
but is physically well motivated. Only for the theories in 
which the action in the Einstein frame exists we can find 
the relation between the thermodynamical quantities in 
the two frames, as we have done above. 



V. APPLICATION TO f{R) THEORIES 

In this section we apply the formulas of the horizon 
entropies in the Jordan frame to inflation and dark energy 
in f{R) theories. In particular the evolution of 5* and S 
will be discussed during inflation (and reheating) in f(R) 
theories. We also study how the horizon entropies evolve 
during an epoch of the late-time cosmic acceleration in 
f{R) dark energy models. In the following we assume 
the flat FLRW space-time (K ^0). 



A. Inflation 

It is known that cosmological inflation can be real- 
ized by the Lagrangian density of the form f{R) = 
R + ai?" {a,n > 0). The first infiation model pro- 
posed by Starobinsky corresponds to n = 2 [s^. Let 
us consider the model f{R) = R + aR^ in the region 
F = df/dR = 1 + naR"-^ > 1. 

During infiation one can use the approximations 
\H/H^\ < 1 and \H/{HH)\ < 1. With these approxi- 
mations, in the absence of matter fluids Eq. reduces 
to 



H 



(n- l)(2n- 1) 



(69) 



This gives the power-law evolution of the scale factor 
(a a t^/'^), which means that inflation occurs for /3 < 1, 
I.e. n > (1 + V3)/2. When n = 2 one has /? = 0, so that 
H is constant in the regime ^ 1. The models with 
n > 2 lead to the super-inflation characterized by _ff > 
and a (x \tQ ~ t|^-'-/l'^l (ig is a constant). 

The standard inflation with decreasing H occurs for 
< /3 < 1, i.e. {1 + V3)/2 < n < 2. In this case 
the horizon entropy p6|) in the equilibrium framework 
grows as S' cx cx during inflation. Meanwhile the 
horizon entropy S — F{R)A/{4:G) in the non-equilibrium 
framework has a dependence S oc R^^^ /H^ cx i/^^"^^) c>c 
^2(2-n) ^j^g regime F ^ 1. Hence S grows more slowly 
relative to 5*. This property can be understood from 
Eq. dm), i.e. 



dS 
lit 



F~dt 



12-Pd,S 
F4-/?"dr 



where 



diS 127r/3(4-/3) 



dt 



G 



HE 



R ■ 



(70) 



(71) 



Here Fji = dF/dR. For the above model the term 
F ~ naR^^~^ evolves as F oc t^^^~"'. This means that 
(l/F)dS'/d< oc t in Eq. ([70]) . which has the same depen- 
dence as the time-derivative of S, i.e. dS'/d< oc t. The 
r.h.s. of Eq. ([TT]) is positive because F^n > and /? > 0, 
so that diS/dt > 0. We have diS/dt oc t3-2n g^^^ hence 
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the last term on the r.h.s. of Eq. ([70)) also grows in pro- 
portion to t. Thus S evolves differently from S because 
of the presence of the term 1/F. 

More precisely each term in Eq. (j70p is given by 



1 2 



dS 
dt 

F~dt 



27r/? 1 

27r/?(2-7i) 1 
G 



1 



B 
1 



F4-/3 dt 



G 



H 



(72) 
(73) 
(74) 



where we have used F » 1. When (1 + v^)/2 < n < 2, 

1. e. < /3 < 1, it follows that dS > 0,dS' > 0,d.,S' > 
for dt > 0. In the limit that n — > 2 the ratio r = 
(2 - n)/{n - 1) of the r.h.s. of Eq. ^ to the r.h.s. 
of Eq. ([71]) approaches 0, so that the entropy production 
term diS gives a dominant contribution to dS. When 
n > 2, i.e. ^ < 0, we have dS < 0,d5' > 0,d^S < for 
dt > 0. Hence the decrease of 5* comes from the negative 
entropy production term d^S*. The entropy S in the equi- 
librium framework decreases for the theories with H > 0, 
whereas S in the non-equilibrium framework can grow 
even in such cases unless the entropy production term 
is taken into account. The equilibrium description of 
thermodynamics allows us to introduce the Bckcnstcin- 
Hawking entropy that mimics the property in General 
Relativity. 

In the Starobinsky's model f{R) = R + R^/{6M^), 
the presence of the linear term in R eventually causes 
inflation to end. Without neglecting this linear term, we 
obtain the following equations: 

H-^ + 3HH + hl^H = , (75) 

R + 3HR + M^R = 0. (76) 

During inflation the first two terms in Eq. (|75l) can be 
neglected relative to others, which gives H ~ — 
We then obtain the solution H ~ Hi - {M'^/6){t - t,) 
with the Ricci scalar R ~ 12H^ — AP, where Hi is the 
Hubble parameter at the onset of inflation (at t = ti). 
The accelerated expansion ends when the slow-roll pa- 
rameter e = —H/H^ ~ AP/{'6H'^) grows to the or- 
der of unity, i.e. H ~ Ml\f^. The horizon entropy 
S grows as S" cx \Hi ~ (AfV6)(t - tC)\ ~^ whereas S ~ 
(7r/G)[2/(3i?2) + 4/Af2j ^ 47r/(GM2) during inflation 
(H^ ^ M^). Hence the horizon entropy S in the equi- 
librium framework increases faster than S in the non- 
equilibrium one, as in the models with (1 + •\/3)/2 <n < 

2. This property is clearly seen in the numerical simula- 
tions of Fig. n (0 < Mt < 30). 

As long as H^ ^ A/^ both dS and the last term 



on the r.h.s. of Eq. (|48p are approximately given 
by ixM"^ l[iGH^)dt. Meanwhile we have (l/F)dS' = 
7rAf^/(18Gff^)dt, which is suppressed by a factor of 
M"^ /{&H'^) relative to dS. Hence the variation of the 




FIG. 1: The variation of the horizon entropy S = A/{4,G) 
in the equilibrium picture as well as S = FA/ {AG) in the 
non-equilibrium picture for the inflation model ,f{R) = R + 
B? / {6K'P) . We also show the evolution of the number of e- 
foldings = \n{a/ai) from the beginning of inflation. Initial 
conditions are chosen to be R/NP = 305 and i? = 0. The 
entropy S evolves faster than S during inflation (0 < Mt < 
30), but their evolution is similar after inflation in which F is 
close to 1. 



horizon entropy S is mainly sourced by the entropy pro- 
duction term diS during inflation. 

The inflationary period is followed by a reheating phase 
in which the Ricci scalar R exhibits a damped oscillation 
with a frequency M [see Eq. (|76)) ]. The evolution of the 
Hubble parameter during the reheating period can be 
estimated as 



H 



3 3 3 

h -(t - tos) H smM(t - 

M 4^ 4M ^ 



X cos 



M 



{t - tos) 



(77) 



where tos is the time at which H starts to oscillate. Tak- 
ing the time average of the oscillations in the region 
M{t - tos) > 1 it follows that (H) ~ (2/3)(f - tos)'^ 
and hence the Universe evolves as a matter-dominated 
one (a oc (t — ios)^^^)- 

The quantity F = 1 + i?/(3A/^) approaches 1 after the 
end of inflation. Equations ([29]) and (PH]) correspond to 
the standard Friedmann equations during the radiation 
and matter eras. S approaches S after the end of infla- 
tion {F ~ 1), which can be confirmed in the numerical 
simulation of Fig. [1] In this regime the entropy produc- 
tion term (l/F) [(2 - /3) / (4 - P)] diS can be negligible 
relative to dS, so that dS* ~ dS*. There are intervals in 
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which the horizon entropies decrease because of the oscil- 
lation of -ff , but the important point is that both 5* and 
S globally increase in proportion to {H)^'^ cx (i — ios)^- 
The entropy production term d^S" is a dominant contri- 
bution to dS during inflation, but after inflation it begins 
to be suppressed relative to dS by a factor of the order 
(lf)^/M^ 1). This shows that it is more convenient 
to take the equilibrium framework in terms of the sin- 
gle horizon entropy S rather than the non-equilibrium 
framework that separates the horizon entropy into two 
contributions. 



B. Dark energy 

Let us next proceed to f{R) dark energy models consis- 
tent with both cosmological and local gravity constraints. 
We focus on models in which cosmological solutions have 
a late-time de Sitter attractor at i? = i?i (> 0) satisfy- 
ing the condition Rf^R = 2/. For the stability of the de 
Sitter point we require that (sH-fs^l 



0<m(i?i)<l, m{R) 



Rf.RR 
f,R 



(78) 



where J^rr = d^f/dR^. Here the quantity m charac- 
terizes the deviation from the ACDM model (/(i?) = 
R-2A). 

There are several other conditions that viable f{R) 
dark energy models need to satisfy: (i) Jr > for 
R> Ri to avoid ghosts, (ii) /^rr > for i? > i?i to en- 
sure the stability of cosmological perturbations [1^ and 
to realize a matter-dominated epoch followed by the late- 
time cosmic acceleration [H], (iii) m rapidly approaches 
-1-0 for R ^ Rq {Rq is the cosmological Ricci scalar to- 
day) to satisfy local gravity constraints [s^]- In other 
words the models need to be close to the ACDM model 
in the region R ^ Rq. More precisely, we require that 

HR) ^ 10^^^ for R « 10^i?o 

A number of authors proposed viable models consistent 
with the above requirements [13, [39l - l46| . One represen- 
tative model is |42| 



f{R) = R^\Rc 



(79) 



where A, i?c, and n are positive constants. For A = 0(1), 
Rc is of the order of Rq. Another similar model is f{R) = 
i?-Ai?c(i?/-Rc)^"/[(i?/i?c)^" + l] mi, which has the same 
asymptotic form f {R) ~ R-XRc[l - {R/Rc)~^''] as that 
in the model ([7^ . When n > 0.9 these models are con- 
sistent with local gravity constraints due to the rapid 
decrease of m for increasing R [43] . 

A simpler f(R) model that has only two free parame- 
ters A and Rc is jil ] 



fiR) = i?-Ai?ctanh( — 

. Rc 



2.0 



1.5 



1.0 



0.50 



0.0 




-1 



-0.5 





z 



0.5 



FIG. 2: The evolution of the horizon entropies S — A/ (AG) 
and S — FA/ (AG) versus the redshift z for the dark energy 
model (|79p with model parameters n — 1 and A — 1.8. The 
entropy S is normalized to be 1 today (z = 0). The initial evo- 
lution of S is similar to that of S, but the difference appears 
as the quantity F deviates from 1. 



in which m ~ 8A(i?/i?c)e"^^/^'= in the region R > Rc. 
For increasing R the quantity m approaches +0 even 
faster than in the model ([751) . Another similar model 
is f{R) = R ~ XRcil - e"^/^") in which case 

TO ~ \{R/Rc)e-^/^'' for R > Rc. 

For the viable f{R) models mentioned above the quan- 
tity F is close to 1 in the region i? ^ i?c, so that the 
evolution of the horizon entropy S is similar to that of S 
for the redshift z = ao/a — 1 S> 1 (ag is the scale factor 
today). The deviation from the ACDM model appears 
for low redshifts {z < 1), which leads to the difference 
between S and S. Since J^rr > for i? > we have 
F < and hence F < 1 provided that R decreases with 
time {R < 0). This means that S should be smaller 
than S for low redshifts, which can be confirmed in the 
numerical simulation of Fig. [5] for the model ((7^ . 

In the following we study the evolution of S and S 
in more details. The horizon entropy S oc in the 

equilibrium picture increases as long as H continues to 
decrease toward the de Sitter attractor. The stability of 
the de Sitter point given in Eq. ((75|) can be divided into 
two cases: (a) stable spiral for < m(Ri) < 16/25 and 
(b) stable node for 16/25 < m{Ri) < 1 [sj].! 



(80) 



^ This comes from the fact that the eigenvalues for the 3x3 ma- 
trix of perturbations about the de Sitter point are given by —3, 
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In the case (a) the solutions approach the attractor 
with the oscillation of R, whereas in the case (b) the 
oscillation of the Ricci scalar does not occur around 
R = Ri. In the former case the horizon entropy S fi- 
nally approaches a constant value at the de Sitter point 
with small oscillations. The numerical simulation in 
Fig. m corresponds to this situation with m{Ri) = 0.358, 
which shows that the oscillation of S around the redshift 
— l<z<— 0.8is really tiny. In the case (b) we have nu- 
merically checked that such oscillations of S disappear, 
as expected analytically. We have carried out numeri- 
cal simulations for other viable /(i?) models such as ([50)1 
and found that the above properties also persist in those 
models. Thus the horizon entropy S globally increases 
with time apart from small oscillations that appear for 
the case < m{Ri) < 16/25. 

Let us estimate the contribution of dS and diS to dS 
in Eq. pS)) . We shall consider the cosmological epoch 
in which the quantity H/H^ is approximately constant, 
that is, H/H^ ~ — (3/2)(l + Wctf), where Wcs is an ef- 
fective equation of the system {wcs = 1/3,0,-1 during 
radiation, matter, and dc Sitter eras, respectively). Then 
the last term in Eq. (|48p is approximately given by 



d,S 



1 2H^ 



H 



F 4ff 2 + H 



d,5: 



27r H Rf^RR 
G f.R 



dt. 



(81) 



Using d5 = -{2Tr/G){H / H^)dt, the entropy production 
term can be simply expressed as 



diS ~ m dS . 
It then follows from Eq. (gS]) that 

dS'/i^ ~ (1 - m) d5 . 



(82) 



(83) 



As long as TO <C 1 the entropy production term in 
Eq. ([82]) is negligible relative to dS*, so that dS ~ 
(l/i^)dS'. As the deviation from the ACDM model ap- 
pears, i.e. TO > 0(0.1), the entropy production provides 
an important contribution to dS. 

As long as the stability condition (|78p is satisfied, the 
deviation parameter has been in the range < to < 1 un- 
til the solutions reach the de Sitter attractor. Provided 
that dS > 0, we then have diS > and dS > from 
Eqs. (|82|) and ((83)) . In fact the growth of S can be con- 
firmed in Fig. [21 apart from the tiny oscillations around 
the dc Sitter attractor. 



VI. CONCLUSIONS 

In the present paper, we have studied thermodynam- 
ics on the apparent horizon with area A in the FLRW 



-3/2 ± v'25 - 16/m{_Ri)/2. 



space-time for modified gravity theories with the La- 
grangian density f{R, 4>, X). If we define the energy mo- 
mentum tensor of "dark" components other than perfect 
fluids as Eq. ([T^ with the Einstein equation (|14p . the 
corresponding energy density ([TT|) and the pressure p2)) 
do not satisfy the standard continuity equation for the 
theories in which the quantity F = df /dR is not con- 
stant. Introducing the Wald's horizon entropy in the 
form S = AF/{AG) associated with a Noether charge, 
we have derived the first-law of thermodynamics given 
by Eq. (j27p in the presence of a non-equilibrium entropy 
production term diS. This non-equilibrium picture of 
thermodynamics arises for the theories with dF ^ 0, 
which include f{R) gravity and scalar-tensor theories. 

If we define the energy density pd and the pressure 
Pd of dark components as in Eqs. ([51]) and ([5^ respec- 
tively, we obtain the standard continuity equation (|33p . 
This corresponds to the introduction of the energy mo- 
mentum tensor T^? as Eq. (|46p with the Einstein equa- 
tion (j45p . Introducing the Bekenstein- Hawking entropy 
in the form S = A/ {AG) , we have found that the first 
law of equilibrium thermodynamics follows from Einstein 
equations. Note that this is different from the approach 
taken in Ref. [Tsj for the realization of equilibrium ther- 
modynamics in which a modified Misner-Sharp mass Ai 
was introduced with the definition of the horizon entropy 
S = AF/{AG). 

The horizon entropy S in our equilibrium framework 
is analogous to that in Einstein gravity. We note, how- 
ever, that this equilibrium thermodynamics in the Jordan 
frame is not in general identical to that in the Einstein 
frame. For scalar-tensor theories with the action ([50)) 
we have derived the explicit relations between thcrmo- 
dynamical quantities in two frames. Our equilibrium de- 
scription of thermodynamics in the Jordan frame is valid 
even for modified gravity theories in which the Einstein 
frame action does not exist. In addition the Jordan frame 
should be regarded as a physical one because of the con- 
servation law of baryons. Hence the direct construction of 
equilibrium thermodynamics in the Jordan frame makes 
much more sense relative to that in the Einstein frame. 

In the flat FLRW background the horizon entropy S 
is proportional to H~^, which grows for decreasing H. 
In other words the violation of the null energy condition 
(pt + Pt > 0, where pT and Pt are the total energy 
density and the pressure respectively) can lead to the de- 
crease of S. We have applied our formalism to f{R) infla- 
tion models with the Lagrangian density f{R) = R+aR^ 
with {^/3 + l)/2 < n < 2 and showed that S globally in- 
creases apart from the oscillation in the reheating phase. 
The global increase of S also persists in f{R) dark en- 
ergy models that satisfy cosmological and local gravity 
constraints in which the solutions approach a de Sitter 
attractor. 

As we have derived in Eq. ([^5)) . the variation of S 
can be expressed in terms of dS in the non-equilibrium 
framework together with the entropy production term 
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diS. The last term in Eq. pSj) . which we denote diS, 
can be important for the models in which the quantity F 
departs from 1. In f{R) dark energy models, for exam- 
ple, it follows that djS ~ mdS and dS / F ~ (1 — m)dS, 
where m = Rf^Rnl f,R corresponds to the deviation pa- 
rameter from the ACDM model. As long as m ^ 1 
the entropy production term can be negligible such that 
dS ~ dS/F, but its contribution to dS becomes impor- 
tant for m > 0(0. 1). The transition from the former to 
the latter regime indeed occurs at low rcdshifts for viable 
f{R) dark energy models. 

We have thus shown that the equilibrium description of 
thermodynamics in the Jordan frame is present for gen- 
eral modified gravity theories. The equilibrium descrip- 
tion of thermodynamics is useful not only to provide the 
General Rclativistic analogue of the horizon entropy irre- 
spective of gravitational theories but also to understand 
the nonequilibrium thermodynamics deeper in connec- 
tion with the standard equilibrium framework. It will be 
of interest to apply our formalism to dark energy dom- 



inated universe by taking into account the entropies of 
dark energy as well as matter inside the horizon along 
the hues of Ref. [H. 
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